Neptune example
epochs = 10
batch_size = 128
lr = 0.0001
optimizer = Adam(lr = lr)
loss = "binary_crossentropy"
PARAMS = {
'epochs' : epochs,
'batch_size' : batch_size,
'lr' : lr,
'optimizer' : optimizer,
'loss' : loss,
}
tags = []
with neptune.create_experiment( name='VGG19',
params=PARAMS,
description='Using VGG19 model and weights to predict CIFAR-100',
tags=tags,
upload_source_files=[]) as npt_exp:
callback = [
# tf.keras.callbacks.EarlyStopping(monitor='val_loss', min_delta=0.005, patience=5, verbose=0, restore_best_weights=False)
ModelCheckpoint('best_model.h5', monitor='val_loss', save_best_only=True),
NeptuneMonitor(npt_exp),
]
X_train, X_test, Y_train, Y_test = [],[],[],[]
model = Model()
model.compile(loss=loss,optimizer=optimizer, metrics=['accuracy'])
history = model.fit(X_train, Y_train, validation_data=(X_test, Y_test), batch_size=batch_size,
epochs=epochs,
verbose=1, callbacks=callback)
results = model.evaluate(X_test, Y_test, batch_size=batch_size)
names = model.metrics_names
npt_exp.set_property(names[0], results[0])
npt_exp.set_property(names[1], results[1])
model.save_weights('model_weights.h5')
npt_exp.log_artifact('model_weights.h5')
npt_exp.log_artifact('best_model.h5')As we can see above, Train set has 50,000 records and the Test set has 10,000 records.
Number of unique targets is 100
Each sample contains a simgle Image and its classification
Let's take a look at some examples from the Trainset
We are using data from the Keras.datasets, thus the data is ready for use.
We can use augmentation, rotations, movements and croping that can be applied here.
But as we can see, the given images are well croped and the main element is centered therefor there is no need for augmentation for this dataset.
Edit - Later while working on the model's improvements, found out that augmentation is helpful (more details at 2.d)
The data is perfectly balanced - Each target has the exact same number of records ( 500 for the train, 100 for the test )
We can see that the top 4 methods were able to get an accuracy score above 90%
Note that most of them were using 'Extra training data', this was not common in previous methods ( from #6 downwards )
Hard separables might be caused by similar backgrounds and\or colors of elements in the pictures, as well as similar shapes of the objects.
Easy separables can be seen as images with different structures ( colors, texture, backgrounds)
Input shape
4D tensor with shape:
if `data_format` is `"channels_first"` - `(batch, channels, rows, cols)`
if `data_format` is `"channels_last"` - `(batch, rows, cols, channels)`We use validation_split of 0.2
Train size will be 80%, Test size is 20%
Loading model...
Confusion matrix
By looking at the examples, even humans might find it difficult to classify the first row. Trees look alike and it's hard to get the correct one (same for Boy-Man and Squirrel-Rabbit)
The second row doesn't have any elements that might be similar between expected and actual prediction
Accuracy on testset was very similar to the validation accuracy, even a bit better
On validation: 36% | On testset: 38.7%
loss_function = sparse_categorical_crossentropy
Prioritize improvements in a list
In the first atempt to improve we will change the following
Confusion matrix - improved
https://keras.io/preprocessing/image/
Example here: https://www.youtube.com/watch?v=14syUbL16k4
Improve graph
As we can see, even though the improved model got better results - it's still overfitting
The red line represents the basic_model's validation status
In order to overcome the overfitting, we will try to add more samples to the training data
We will do so by setting the validation_split to 0.9
In addition, we add ImageDataGenerator for augmentation usage
Augmentation parameters
Confusion matrix - Improved with augmentation
Much better, we can see that both training and validation are more correlated
previous val_loss was 2.4 and val_accuracy was 0.44
----------- Training score -----------
Accuracy = 0.52 | Loss = 1.75
----------- Validation score -----------
Validation Accuracy = 0.48 | Validation Loss = 2.01
----------- Prediction score -----------
Accuracy on test set is: 48.76%
The red line represents the basic_model's validation status
Keras pre-trained models https://keras.io/applications/
ReduceLROnPlateau info
Fit the model and present metrics
Evaluate the results
As seen in the graphs above, the model is overfitting
Load VGG model
Confusion matrix - VGG19
Fit RandomForest Classifier
Results of using VGG19 model as feature extraction
The classifier is less accurate than the VGG model's accuracy ( 40% instead of 50% )
Improves
Increase the image size ( Due to RAM limit, we will increase to 96x96 )
Add layers to the model
Fit using more epochs
Getting a much cleaner image
Resize train and validation images
Fit model with Neptune
Load transfer model VGG19
Resize images
Confusion matrix - VGG19 with resize
Classify using Randomforest